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1. The practical utility and the limits of authoring-svs terns 

Are authoring-systems really useful? The answer to this question, 
asked many times and in many places already, of course depends exclusively on 
the quality of the programs. In fact, given a certain type of amateurish 
educational software in Europe—and elsewhere--and especially its deplorable 
quality, one could as well put the question in this way: is CALI~or 
CALL — really useful? 

Actually, I just want to say that only real quality software is 
worth ev;^luating. And when I apply the term quality-sof tware to 
CALI-software, I mean that it must come up to the highest pedagogical, 
didactical, technical and financial expectations, and that all-round 
user-friendliness is paramount! 

Some researchers pretend that authoring-systems are not useful, 
period. Such an absolute assertion cannot be taken seriously: all too often the 
very authoring-systems criticized have not been examined and tested with regard 
to their pedagogical, didactical, technical and financial qualities and their 
user-friendliness! In fact, it is very -aasy to criticize whatever educational 
software package: use a package in a wrong application, and it can be qualified 
as a bad one. Paraphrasing the Sinatras, I should say: "Boots are made for 
walking" — not for swimming. 

The quality of authoring-systems , therefore, must be evaluated with 
regard to various parameters, such as: 

- the specific applications for which they are designed and developed; 

- the public for whom they are meant; 

- the level at which they must be used; 

- the available hardware; 

- the familiarity of potential users with data processing applications; 

- the familiarity of potential users with CALL-and CALI-applications; 

- etc. 

In a word, authoring-systems are too often evaluated as if they were 
universal CALL- or CALI-systems, as if they could be used for any educational 
purpose—which is impossible. The same holds for a manual. A manual can be very 
good and useful within a specific discipline at a certain level, but 
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inconsistent or incomplete at another level, even useless for some applications 
within or outside the discipline concerned. It must be admitted that the first 
authoring-systems that were published were too often presented as the final and 
absolute solution to all problems in CALI... I think that in educational 
matters— and especially in computer assisted instruction—there will never be a 
final solution. Moreover, there's nothing more dangerous than a computer 
freak in a classroom — whether it be a student or a teacher — the latter is 
perhaps the more dangerous. 

Let us stop considering the computer as that final solution for 
everything; let us instead take an honest but hard look a what that marvellous 
machine can do for us teachers, who want to use all available and effictive 
means to make intelligent and competent people of our students or pupils.,. 

In the history of educational software, one generally distinguishes 
the following phases: 

a. In the initial phase, that of single programs , a programmer had to write 
different programs for each exercise or for each lesson; very often, the 
programmer and the teacher were different individuals who did not always 
really understand each other's pedagogical and technical problems; 

b. This was the major reason why so-called authoring languages were 
developed. The principal goal of these authoring-] anguages was to enable the 
teacher himself to make lessons and exercises. It was claimed that 
authoring-languages were simplified programming languages which would indeed 
allow the teacher to write his own programs. But experience has proved that 
teachers had great difficulty writing programs, even in an 
authoring-language, which seemed to be too complex a system to be learned 
and exploited efficiently by most teachers. In fact, the authoring-language 
was often nothing more than just another programming language... 

c. This is why authoring-systems were developed. An authoring-system is a 
software package whose major characteristic is great user-friendliness. The 
teacher does not need to know anything about data processing: he generally 
works in a menu-driven and conversational technique to make lessons or 
exercises. It is an irrefutable fact that authoring-systems—especially the 
good ones! —have been at the basis of the spread and popularity of 
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But it is precisely this us^r-friendliness that constitutes a danger in 
the evaluation of authoring-systems. Users often think that an authoring-system 
can be used for any educational purpose • People, especially inexperienced 
people, often have exaggerated expectations of data-processing applications and 
expect their computers to "know" and to "do" everything. Some people have 
already learned that this is not really the case*.. 

An authoring-system still demands some work of the teacher: he must 
still make the lesson or the exercises himself. Some (lazy) teachers prefer a 
kind of ready-made programs or exercises they only have to put into a disk 
drive. They forget that the range of applications and possibilities of an 
authoring-system is much more broad^ than that of a ready-made software 
package, however good that may be. 

Exaggerated expectations in^ and failure to exploit all the 
possibilities of, authoring-systems are two of the main reasons why people 
often consider them to be useless. 

However, I myself as well as several of my colleagues at the University 
of Antwerp have learned that an authoring-system CAN be useful, provided that 

a. the authoring-system is not used outside of the range of applications for 
which it has been developed; 

b. a serious and honest evaluation of the real uses and of the results of the 
authoring-system is made; 

c. ail the criticims of teachers and students or pupils are taken into account; 

d. one is prepared to correct, adapt, extend and even substantially modify the 
authoring-system on the basis of a great amount of practical experience. 

That is what we tried to keep in mind while working with my 
authoring-system DRILL, which I would now like to present. 

The authoring-system DRILL; general characteristics 



DRILL is a multifunctional authoring-system specially designed for 
computer-assisted language learning (CALL) and computer-assisted language 



instruction (CALI)» It contains three principal modules: 

a. a MANAGER, which allows the teacher to write and organize exercises; 

b* a DRILLER, which allows the student/pupil to do them; 

c. an ANALYZER, which tabulates the student's/pupil's results, thus allowing 
the teacher to evaluate both his students/pupils and the effectiveness of 
his exercises • 

(a) and (c) are protected by passwords. 

DRILL is language-independent, which means that it can be used for all 
languages for which a specific character set is available on a PC, e.g., Dutch, 
French, English, German, Latin, Spanish, However, DRILL can also be used in 
other disciplines. In fact, different schools in Flanders and in the 
Netherlands also use it in geography, history, etc. 

DRILL can be used at all educational levels, from primary school to 
university. Users of the system must be able to work on a traditional keyboard, 
but chey do not have to know anything about data processing. They only must be 
told how to put a floppy into a disk driveand how to switch on the computer. 
After that everything is menu-driven. 

DRILL has now been used at the University of Antwerp (UFSIA, 
Departments cf Romance and Germanic Philology) for more than 5 years. The 
system was developed in different phases; the current version is release 4.0. 

Using DRILL requires a PC with minimum 64 Kb RAM (at the University of 
Antwerp we have versions running on APPLE He, BURROUGHS 20B, IBM-PC 
(compatibles and clones) and IBM PS/2.), one disk drive (and naturally MS-DOS 
for IBM and compatibles, Apple-DOS for the APPLE He, or BTOS for the BURROUGHS 
B20), a monochrome screen (24 lines and 80 columns), and the DRILL floppy. A 
printer is desirable but not necessary. 

The MANAGER 

This module contains different submodules, of which the most important 
are the ones enabling the teacher to make exercises and the one enabling him to 
define the parameters and scenarios of the students * exercises. 
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For each question the teacher wants to put into the system, he has to 
type in (two or) three blocs of information • The first two blocs — DATA and 
QUESTION (the two may be collapsed) — are those which the student will see on 
the screen when he does his exercise. In these two blocs the student must be 
given all the necessary information to find his answer. But the teacher is 
absolutely free to put whatever information in DATA and/or QUESTION. The third 
bloc must contain the right answer(s) (maximum 8). Some examples: 

French 

DATA : II faut que vous ... davantage. 

QUESTION : Mettez TRAVAILLER a la forme correcte. 

ANSWER : travailliez 

German 

DATA : Er ist ... 

QUESTION : Fill in and translate: of the first rank 

ANSWER : ersten Ranges 

English 

DATA : His feelings are "very much" hirt. 

QUESTION : Replace "very much" by an adverb 

ANSlffiR : deeply 

Dutch 

DATA : Hi.j ... deze vraag perfect. 

QUESTION : BEANTWOORDEN , O.V.T. 

ANSWER : beantwoordde 

In actual practice, most teachers put the same type of exercises on one 
floppy (and on the numerous copies they make): a floppy with exercises on verb 
conjugation, one on the placement of adverbs, one on the morphology of the 
adverb, one on the French participe passe, one on the morphology of French 
or Dutch or German nouns and adverbs, and so on. Here are some examples of the 
numerous possible applications: 

a. ENGLISH: must, have to, ought, etc. 

b. FRENCH: accord du participe pass{ 

c. GERMAN: nominal declination 
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d. DUTCH: spelling problems 



e. All 


languages: 


conjugation of verbs 


f. 


• 
• 


the use of prepositions 


8* 


• 
• 


morphology of the adverb (not in Dutch or German) 


h. 


• 
• 


vocabulary 


i. 


• 
• 


morphology 


j. 


• 


fill in~exercises 


k. 


• 
• 


aetmxtions of grammatical terminology 


1. 


• 


expressions and idioms 


m. 


• 


translations 


etc. 







The system allows the teacher to correct, change, adapt, modify, and 
delete any question, both while making the exercise and afterwards. It 
automatically writes the (modified) questions on the floppy and assumes the 
management of all the data (with a maximum of 250 questions on one floppy). 
Naturally it is not necessary to write all one's questions in one session: this 
can be done in different sessions over several days, weeks or evan months. But 
the teacher can always get an overview of what is on a floppy (either on screen 
or on hard copy). 

The MANAGER also allows for the initialization of a floppy. When a 
class or a group of students have done several exercises with a floppy, the 
teacher can re-initialize the floppy (and the copies) in two ways. 

1. A complete initialization means that the teacher erases all the data he 
or the student/pupil on the floppy. After such an initialization, he will 
have a "new empty" floppy on which he can make other kinds of exercises. 

2. In ai partial initialization the teacher retains the questions on the 
floppy but erases the error data, i.e., all the students' and error counters 
are put back to zero. After such an partial initialization, the floppy will 
look as if no student had ever done any exercise on it. 

The reader will already have realized that a school needs only a single 
floppy, simply because this master can be copied endlessly— with or without 
exercises and/or error data on it. 
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Another very important submodule of the MANAGER is called PARAMETERS • 
This allows the toacher to define which scenario he will choose for the 
students' exercises. The following parameters can be selected • 

1* The teacher can ask a student to input his name whenever he does an 
exercise. This allows the ANALYZER to make an error analysis for each 
individual student and enables the teacher to see what kind of errors a 
specific student has made, ilowever, the teacher can also select "Anonymous" 
for simple drilling, when he does not need to know which students have made 
this or that kind of error, 

2, The teacher can define how many questions a student must answer in one 

session (minimum 5 and maximum 30), but he can also let the students choose* 

3* The teacher can define the language version of the DRILLER: each floppy of 
DRILL has English, French, Dutch and German versions. Selecting, e.g., the 
English version, means that all comments and instructions in the DRILLER 
will be given in English. 

4. We will see (in I.2.) that a student can have up to four chances to give his 
answer. The teacher decides which score he will give for a good answer on 
the first, tho second, the third and the fourth attempt — and/or, if he so 
wishes, how many marks he will subs tract for each wrong answer on these four 
attempts. He can also simply give 0 for a wrong answer. 

5. The teacher can activate or deactivate the error typing routine, which 
allows the student to correct his answer before the system considers it 
definitive. 

6. The teacher can make any question on the floppy available or unavailable. 
Making a set unavailable means that the question remains on the floppy, but 
that a students working with the DRILLER will not get it until the teacher 
has made it available again. This also means that the teacher can make a 
series of, e.g., 250 questions of which he only makes 100 (easy or specific) 
ones available in the first phase. 

7. When a student has made an error, the teacher can tell the system to print 
the right answer on the screen (feedback). But he can also refuse to show 
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the right answer — for instance, in examinations or tests. 

8. When then teacher has selected the feedback option—he can activate a 
rewriting routine, which requires the student to copy the right answer on 
the keyboard • 

9. The last parameter is a very important one: the teacher can select the help 
mode of the exercises. 

9.1. In mode 1, the student gets no help at all from the system. He simply must 
type in his answers. 

9.2. In ^ode 2, the scudent does get help (i.e., correct answer^) from the 
system, but whc-n he makes errors, they are printed onscreen. 

9.3. In mode 3, the student also gets the correct answers from the system, but 
only those, even when he types in a wronp, character or answer. In other 
words; this mode never shows a wrong anr^wer never appears onscreen! 

All these parameters can be reset at all times. 
The DRILLER 

As we have already explained, the details of the scenario of the 
DRILLER depend on the way the parameters have been set by the teacher. But the 
fundamental structure of the exercise is as follows: 

a. Phase _1 

For each question, the student sees DATA and/or QUESTION on the 
screen and has to type in an answer. If the answer is right, he gets the 
following question. If the answer is wrong, he will get another opportunity 
to ^^'ve the correct answer. If the answer is* wrong again, it will depend on 
the parameter settings whether he gets the right answer and whether he has 
to copy it. 
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Phase 2 



There will be a phase 2 whe:i the student has made at least one error 
in phase 1, Each question which the student has answered incorrectly in 
phase 1 will be presented again in phase 2 (following the same scenario as 
in phase 1). 

c. Closing phase 

After having done phase 1 and 2, the student gets his score and, if 

he wants one, a hard-copy print-out of the questions to which he has given 

wrong answers. After that he can stop or go on to do another session. 



5. The ANALYZER 

The student does not see that his results and error data are 
automatically written on the floppy. (Of course he can be told.) These data 
will be the basis of the error analysis. The following data, among others, can 
be produced by the system. 

In the general error analysis , pertaining to all students, the 
system will give: 

1. the number of times that the system selected (at random) all questions; 

2. the number of correct answers to each of these questions on the 1st, 2nd, 
3rd and 4th attempt; 

3. the number of wrong answers to each of these questions on the 1st, 2nd, 3rd 
and 4th attempt; 

4. the percentage of right and wrong answers to each question; 

5. the overall percentage of right and wrong answers. 

In the selective error analysis the teacher can ask for an error 
analysis for any question he wants, by typing in the number of that question, 
or by typing in part of the text of the question (even one single word). 
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In the degressiv e error analysis the system will classify all the 
sets in the following order: at the top of the list, the te::cher will find ihe 
set on which the students made the most errors (e.g. 100 % errors), then the 
set with 99 Z errors, then 98 Z and so on, down to a percentage the teacher 
defines himself. So, he can ask a degressive error ans^ysis of all the sets on 
which the students made an average percent of errors of 50 Z. 



In the degressive error analysis the system will classify the 
questions in descending order of difficulty: from the question that elicited 
the highest number of errors (e.g., 100%), down to a percentage the teacher can 
define himself. Thus he can ask for a degressive error analysis of all the 
questions to which the students gave an average percentage of wrong answers of 
50 Z. 

Of course, the teacher can also ask for a list of the names of all the 
participants—arranged alphabetically or not — \»lth their scores. 

Each error analysis can be printed out onscreen or on hard copy. 



6. Some didactic and pedagogical aspects 

a. The fact that the system is "empty" when he starts allows the teacher to put 
into the authoring-system nearly any type of exercise he wants. DRILL is an 
educational software package meant for teachers who do not belong to the 
category of the lazy ones... However, this does not mean that making an 
exercise battery with DRILL takes a lot of work—not at all. And one must 
not forget the almost absolute freedom he has in the choice, the content and 
the form of the exercises he wants to make. 

b. The numerous parameters which the teacher can set allow him to adapt the 
DRILLER to the level of his students, to the content and type of his 
exercises, to their applications (simple drill, test, examination, etc.), to 
the mother tongue of his students, etc. 

c. Experience has taught me and my colleagues that students like working with 
this system very much. They generally prefer doing DRILL-exercises to doing 
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traditional exercises on a sheet of paper — which means that, for certain 
kinds of exercises which can be efficiently automated in DRILL, the 
motivation of students is very high. 



d. The possibility of immediate feedback makes of DRILL not only a simple drill 
and practice-system, but also a real learning-systeni. Something similar 
holds for the three help-modes: when students get help during their 
exercises, they can really learn something. 

e. The error analysis provides extremely detailed information about the 
students^ proficiency, about their attendance and progress (which students 
have come to work with DRILL, which students have done enough exercises), 
about the degree of difficulty of the questions, etc. In fact, the 
information provided by the error analysis can be used by the teacher as a 
basis to organize and define the contents of his traditional (non-computer 
assisted) lessons and to differentiate individual educational procedures in 
the classroom. As a matter of fact, the greatest surprise for most teachers 
is provided by the results of the ANALYZER, which gives them a type of 
statistical data they are not used to. 

f. Perhaps the most important conclusions about DRILL are: 

1. that the system has now been used continuously for more than 5 years by 
all Kinds of teachers and professors, including in particular teachers 
who had never worked with a computer before, but who nevertheless 
mastered it very easily; 

2. that DRILL is being used at different educational levels (in Flanders and 
in the Netherlands, from primary schoo^ to university, and also in adult 
education); 

3. that DRILL, though initially developed for CALI, is useful and effective 
in other disciplines and matters; 

4. that DRILL is now used primarily m CALI-exercises in English, French, 
Dutch, German and Latin; 

5. that the following stuation typically obtains: two or three students 
working at an exercise, at a computer in or outside the classroom, which 
in fact already constitutes an interesting conversational situation, 
motivating the students to discuss the exercise, the results, the 
feedback, etc.; 

6. that DRILL has already been substantially adapted and modified several 
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times account being takea of the remarks of the users- -prof essors, 
teachers, students and pupils — so that there is now a completely revised 
release 4*0 and that even a release UA* is already being prepared. (Note 
that users always get a free copy of each new release.) 

Please, do not tell to all these enthusiastic people that an 
authoring-system is useless, that you cannot do anything interesting with an 
authoring-system, that it is too complex to be used by teachers or 
students — and I could imagine a whole series of other criticisms. But, before 
people talk about whatever educational system, tell them that they should first 
look honestly and objectively at these systems, or rather, that they should 
first use them with their students. 

There are really a lot of prejudices concerning CALX, both for and 
against. In fact, authoring-systems—and I would not know why DRILL should be 
an exception — are not at all universal systems allowing you to do almost 
any::hing. On the contrary, each authoring-system—just like a manual — has its 
limits, which the teacher has to get to know and to define. Moreover, I am the 
first to agree that a teacher must not try to do absolutely everything on 
computers. There are a lot of procedures — especially in language 
instruction — that cannot reasonably be done on a computer. Too great 
enthousiasm is as bad as too critical an attitude, and perhaps a lot more 
dangerous.,, — but that's another problem! 
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